Comparison of Selected Model Evaluation Criteria for Maintenance Applications

Model Based Preventive Maintenance relies on creating models that can either predict future operating states or upcoming failures directly. Since no modeling algorithm can guarantee a best solution in every situation, it becomes necessary to evaluate the solutions generated by these techniques. This paper reviews some popular criteria traditionally employed in model evaluation. Several evaluation criteria proposed in the literature are restricted in their applicability because of their assumptions about the modeling process/data. Some evaluation criteria are tested on two artificial datasets. The results from our tests indicate that Akaike Information Criterion (AIC) has superior performance. The conclusion has been used and verified in one industrial monitoring application. Keyword: Maintenance, Model Selection, PRESS, AIC and 2 R * Corresponding author: J. Shi, 630 Rhodes Hall, University of Cincinnati, Cincinnati, OH 45221. Tel: 513 2212885. Fax: 513 556-3390. Email: [email protected].